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Abstract 



A completion of an m-by-« matrix A with entries in {0, 1,*} is obtained by setting all ^-entries 

to constants and 1 . A system of semi-linear equations over GF2 has the form Mx — f{x), where 

S,*/ I M is a completion of A and / : {0, 1}" ^ {0, 1}™ is an operator, the /th coordinate of which can 

S^ ■ only depend on variables corresponding to ^-entries in the /th row of A. We conjecture that no such 

c/3 I system can have more than 2"^*^™'-^) solutions, where e > is an absolute constant and mr(A) 

O . is the smallest rank over GF2 of a completion of A. The conjecture is related to an old problem 

of proving super-linear lower bounds on the size of log-depth boolean circuits computing linear 

operators x n- Mx. The conjecture is also a generalization of a classical question about how much 

K^ I larger can non-linear codes be than linear ones. We prove some special cases of the conjecture 

O^ i and establish some structural properties of solution sets. 

o 

. _ 1 Introduction 

in 

o 

Q I One of the challenges in circuit complexity is to prove a super-linear lower bound for log-depth circuits 

over {&, V, -1} computing an explicitly given boolean operator / : {0, 1}" — >• {0, 1}". Attempts to solve 
it have led to several weaker problems which are often of independent interest. The problem is open 

'k>( I even if we impose an additional restriction that the depth of the circuit is 0{logn). It is even open for 

;_( ' linear log-depth circuits, that is, for log-depth circuits over the basis {®, 1}, in spite of the apparent 

simplicity of such circuits. It is clear that the operators computed by linear circuits must also be linear, 
that is, be matrix-vector products x — ;• Mx over the field GF2 = ({0, 1}, ®, •), 

An important result of Valiant [27] reduces the lower bounds problem for log-depth circuits over 
{&,V,-i} to proving lower bounds for certain depth-2 circuits, where we allow arbitrary boolean 
functions as gates. 

1.1 Reduction to depth-2 circuits 

A depth-2 circuit of width w has n boolean variables x\,...,Xn as input nodes, w arbitrary boolean 
functions h\,...,hw as gates on the middle layer, and m arbitrary boolean functions g\,...,gm as 
gates on the output layer. Direct input-output wires, connecting input variables with output gates, 
are allowed. Such a circuit computes an operator / = (/i , . . . ,/,„) : {0, 1}" — )■ {0, l}™ if, for every 
/ = 1 , . . . , m, 

fi{x) =gi{x,hi{x),...,h„{x)). 
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The degree of such a circuit is the maximum, over all output gates gi, of the number of wires going 
directly from input variables x\,.. . ,x„ to the gate gi. That is, we ignore the wires incident with the 
gates on the middle layer. Let deg^^,(/) denote the smallest degree of a depth-2 circuit of width w 
computing /. 

It is clear that deg„ (/) = for / : {0, 1 }" — > {0, 1 }" : just put the functions /i ,...,/„ on the middle 
layer. Hence, this parameter is only nontrivial for w <n. Especially interesting is the case when 
w = 0{n/\n\nn) (see also Theorem 2.2 in |20| for more details): 

Lemma 1.1 (Valiant [27]). Ifdeg„{f) = n^^^' for w = 0{n/lnlnn), then the operator f cannot be 
computed by a circuit of depth 0{\nn) using 0{n) constant fan-in gates. 

Recently, there was a substantial progress in proving lower bounds on the size of (that is, on 
the total number of wires in) depth-2 circuits. Superlinear lower bounds of the form Q.{n\og^n) 
were proved using graph-theoretic arguments by analyzing some superconcentration properties of the 
circuit as a graph |il[Tl|T5l[IIl[l6l|2l|20l|2l]|22l. Higher lower bounds of the form Q.{n^l^) were 
proved using information theoretical arguments El [3. But the highest known lower bound on the 
degree of width w circuits has the form Q.{{n/w) \n{n/w)) ||20H . and is too weak to have a consequence 
for log-depth circuits. 

A natural question therefore was to improve the lower bound on the degree at least for linear 
circuits, that is, for depth-2 circuits whose middle gates as well as output gates are linear boolean 
functions (parities of their inputs). Such circuits compute linear operators x i— ;■ Mx for some (0, 1)- 
matrix M; we work over GF2. By Valiant's reduction, this would give a super-hnear lower bound for 
log-depth circuits over {®, 1}. 

This last question attracted attention of many researchers because of its relation to a purely alge- 
braic characteristic of the underlying matrix M — its rigidity. The rigidity ^M{r) of a (0, l)-matrix M 
is the smallest number of entries of M that must be changed in order to reduce its rank over GF2 to 
r. It is not difficult to show (see |[27l ) that any linear depth-2 circuit of width w computing Mx must 
have degree at least MM{w)/n: If we set all direct input-output wires to 0, then the resulting degree-0 
circuit will compute some linear transformation M'x where the rank of M' does not exceed the width 
w. On the other hand, M' differs from M in at most dn entries, where d is the degree of the original 
circuit. Hence, Mm{w) < dn from which d > ^M{w)/n follows. 

Motivated by its connection to proving lower bounds for log-depth circuits, matrix rigidity (over 
different fields) was considered by many authors, [23l[Il[l7l|71ll6ll20l|25l|2llIS[IIlll9l|26l among 
others. It is therefore somewhat surprising that the highest known lower bounds on ^M{r) (over 
the field GF2), proved in |l7]|25l also have the form Q.{{n'^/r)ln{n/r)), resulting to the same lower 
bound Q.{{n/w)\n{n/w)) on the degree of linear circuits as that for general depth-2 circuits proved 
in [20 1 . This phenomenon is particularly surprising, because general circuits may use arbitrary (not 
just linear) boolean functions as gates. We suspect that the absence of higher lower bounds for linear 
circuits than those for non-linear ones could be not just a coincidence. 

Conjecture 1 (Linearization conjecture for depth-2 circuits). Depth-2 circuits can be linearized. That 
is, every depth-2 circuit computing a linear operator can be transformed into an equivalent linear 
depth-2 circuit without substantial increase of its width or its degree. 

If true, the conjecture would have important consequences for log-depth circuits. Assuming this 
conjecture, any proof that every depth-2 circuit of width w = 0{n/\n\nn) with unbounded fan-in 
parity gates for a given linear operator Mx requires degree n^^^' would imply that Mx requires a super- 
hnear number of gates in any log-depth circuit over {&,V,-i}. In particular, this would mean that 



proving high lower bounds on matrix rigidity is a much more difficult task than assumed before: such 
bounds would yield super-linear lower bounds for log-depth circuits over a general basis {&,V,-i}, 
not just for circuits over {©,1}. 

As the first step towards Conjecture [T] in this paper we relate it to a purely combinatorial conjec- 
ture about partially defined matrices — the min-rank conjecture, and prove some results supporting this 
last conjecture. This turns the problem about the linearization of depth-2 circuits into a problem of 
Combinatorial Matrix Theory concerned with properties of completions of partially defined matrices 
(see, e.g., the survey [8|). Hence, the conjecture may also be of independent interest. 

Unfortunately, we were not able to prove the conjecture in its full generality. So far, we are only 
able to prove that some of its special cases are true. This is not very surprising because the conjecture 
touches a basic problem in circuit complexity: Can non-linear gates help to compute linear operators? 
This paper is just the first step towards this question. 

1.2 The Min-Rank Conjecture 

A completion of a (0, 1 , *)-matrix A is a (0, 1 )-matrix M obtained from A by setting all *'s to constants 
and 1. A canonical completion of A is obtained by setting all *'s in A to 0. 

If A is an m-hy-n matrix, then each its completion M defines a linear operator mapping each vector 
X G {0, 1}" to a vector Mx G {0, 1}'". Besides such (linear) operators we also consider general ones. 
Each operator G : {0, 1}" — )• {0, 1}"' can be looked at as a sequence G = (gi, . . . ,gm) of m boolean 
functions g,: {0,1}" ^{0,1}. 

We say that an operator G = (gi, . . . ,gm) is consistent with an m-hy-n (0, l,*)-matrix A = (a,y) if 
the /th boolean function g,- can only depend on those variables Xj for which aij = *. That is, the /th 
component gt of G can only depend on variables on which the /th row of A has stars (see Example 

Mi. 

Definition 1.2. With some abuse in notation, we call a set L C {0, 1}" a solution for a partial matrix 
A if there is a completion M of A and an operator G such that G is consistent with A and Mx = G{x) 
holds for all jc G L. A solution L is linear if it forms a linear subspace of {0, 1}" over GF2. 

That is, a solution for A is a set L of (0, l)-vectors of the form L = {x: Mx = G{x)}, where M is 
a completion of A, and G is an operator consistent with A. A solution L is linear, if x© j G L for all 
x,y G L. 

Since, besides the consistency, there are no other restrictions on the operator G in the definition of 
the solution L, we can always assume that M is the canonical completion of A (with all stars set to 0). 

Observation 1.3 (Canonical completions). If L = {x: Mx = G{x)} is a solution for A, and M' is 
the canonical completion of A, then there is an operator G' such that G' is consistent with A and 
L = {x:M'x = G\x)}. 

Proof. The /th row m,- of M must have the form m, = m^ + pj, where m^ G {0, 1 }" is the /th row of the 
canonical completion M' of A, and pj G {0, 1}" is a vector with no I's in positions where the /th row 
of A has no stars. We can then define an operator G' = (gj , • • . ,g'm) by g'j{x) := gi{x) © {pj,x). (As 
customary, the scalar product of two vectors x,y G {0, 1}" over GF2 is {x,y) = ^"^j Xjji mod 2.) Since 
G was consistent with A, the new operator G' is also consistent with A. Moreover, for every vector 
X G {0, 1}", we have that (m,',x) = gi{x) iff {m'j,x) = g'i{x). D 

We are interested in how much the maximum opt(A) = maXi,|L| over all solutions L for A can 
exceed the maximum lin(A) = max/, \L\ over all linear solutions L for A. It can be shown (Corollarv l6.3l 



below) that 

lin(A) = 2^-^'W , 

where mr(A) is the min-rank of A defined as the smallest possible rank of its completion: 

mr(A) = min{rk(M) : M is a completion of A} . 

If we only consider constant operators G, that is, operators with G{x) = b for some b G {0, 1}'" 
and all X € {0, 1 }" , then Linear Algebra tells us that no solution for A can have more than 2"^'' vectors, 
where r = rk(M) is the rank (over GF2) of the canonical completion M of A, obtained by setting all 
stars to 0. 

If we only consider ajfine operators G, that is, operators of the form G{x) = Hx®b where H is 
an m-hy-n (0, l)-matrix, then no solution for A can have more than 2"^'™'('^) vectors, because then 
the consistency of G{x) with A ensures that, for every completion M of A, the matrix M©// is a 
completion of A as well. 

Remark 1.4. This last observation implies, in particular, that opt(A) < 2"^™^^^' for all (0,1,*)- 
matrices A with at most one * in each row: In this case each g, can depend on at most one variable, 
and hence, must be a linear boolean function. 

We conjecture that a similar upper bound also holds for any operator G, as long as it is consistent 
with A. That is, we conjecture that linear operators are almost optimal. 

Conjecture 2 (Min-Rank Conjecture). There exists a constant £ > such that for every m-by-n 
(0, l,*)-matrix A we have that opt(A) < 2"^'^™'(^) or, equivalently, 

op.(A)<2.(!lfl)'. („ 

Remark 1.5. To have consequences for log-depth circuits, it would be enough, by Lemma [TTl that 
the conjecture holds at least for e = o(l/loglog?i). 

Example 1.6. To illustrate the introduced concepts, let us consider the following system of 3 equations 
in 6 variables: 

Xl ©X6 =X3 -Xs 
X2®X3©X4 =Xi • (X5©X6) (2) 

X4 = (X2 ©X5) • (X3 ©Xe) . 

The corresponding (0, 1, *) -matrix for this system is 

'1 * * 1\ 
* 1 1 1 * * , (3) 

0**1**/ 

and the system itself has the form Mx = G(x), where M is the canonical completion of A: 

/I r 

M= 1 1 10 
\0 1 0/ 



and G = {gi,g2,g3) '■ {0, 1}^ — ?> {0, 1}^ is an operator with 

^l(x) =X3-X5; 

g2{x) =xi -(xsexe); 
gsix) = (X2 ©X5) • (X3 ©xe) . 

The min-rank of A is equal 2, and is achieved by the following completion: 

/I r 

M' = 1 1 1 
\0 1 1 1 Oy 

1.3 Our results 

In Section [2] we prove the main consequence of the min-rank conjecture for boolean circuits: If true, 
it would imply that non-linear gates are powerless when computing linear operators Mx by depth-2 
circuits (Lemmas 12.21 and |231 ). 

In Sections [3] and ID we prove some partial results supporting Conjectures [T] and |2l We first show 
(Corollary 13.41) that every depth-2 circuit of width w computing a linear operator can be transformed 
into an equivalent linear depth-2 circuit of the same degree and width at most w plus the maximum 
number of wires in a matching formed by the input-output wires of the original circuit. 

We then prove two special cases of Min-Rank Conjecture. A set of (0, 1, *)-vectors is independent 
if they cannot be made linearly dependent over GF2 by setting stars to constants and 1 . If A is a 
(0,1,*) -matrix, then the upper bound opt(A) < 2"^'' holds if the matrix A contains r independent 
columns (Theorem 14.41 ). The same upper bound also holds if A contains r independent rows, and the 
sets of star positions in these rows form a chain with respect to set-inclusion (Theorem 14. 111 ). 

After that we concentrate on the structure of solutions. In Section [5] we show that solutions for a 
(0, 1 , *)-matrix A are precisely independent sets in a Cayley graph over the Abelian group ({0, 1}" , ®) 
generated by a special set Ka Q {0, 1}" of vectors defined by the matrix A (Theorem l5.2l ). 

In Section |6] we first show that every linear solution for A lies in the kernel of some completion 
of A (Theorem 16.21 ). This, in particular, implies that lin(A) = 2"^"^('*) (Corollary 16.31 ). and gives an 
alternative definition of the min-rank mr(A) as the smallest rank of a boolean matrix H such that 
Hx 7^ for all x^Ka (Corollary 16.41 ). In Section|7]we show that non-linear solutions L must be "very 
non-linear": if s is the maximum number of *'s in a row of A, and if L contains a linear space V such 
that no nozero vector with s or fewer I's is orthogonal to V , then L is contained in a linear solution 

for A (Theorem ITB. 

In Section [8] we consider the relation of the min-rank conjecture with error-correcting codes. We 
define (0, 1, *) -matrices A, the solutions for which are error-correcting codes, and show that the min- 
rank conjecture for these matrices is true: In this case the conjecture is implied by well known lower 
and upper bounds on the size of linear and nonlinear error correcting codes (Lemma [83] ). 

For readers convenience, we summarize the introduced concepts at the end of the paper (see 
Table [B. 

2 Min-rank conjecture and depth-2 circuits 

Let 7^ be a depth-2 circuit computing a linear operator x — )• Mx, where M is an m-hy-n (0, l)-matrix. 
Say that the (/,7)th entry of M is seen by the circuit, if there is a direct wire from Xj to the /th output 



gate. Replace all entries of M seen by the circuit with *'s, and let Af be the resulting (0, 1, *)-matrix. 
That is, given a depth-2 circuit F computing a linear operator x — ;• Mx, we replace by *'s all entries of 
M seen by the circuit, and denote the resulting (0, 1, *) -matrix by Af. Note that the original matrix M 
is one of the completions of Af ; hence, rk(M) > mi{Ap). 

Lemma 2.1. Every linear depth-2 circuit F has width(F) > rar{Af). 

In particular, if F computes a linear operator x i— ;• Mx and has no direct input-output wires at all, 
then Af=M and width (F) > rk(M). 

Proof. Let Mx be a linear operator computed by F. Every assignment of constants to direct input- 
output wires leads to a depth-2 circuit of degree d = computing a linear operator Bx, where B is 
a completion of Af . This operator takes 2*'^' different values. Hence, the operator H : {0, 1}" — )■ 
{0, 1}"' computed by w = width(F) boolean functions on the middle layer of F must take at least so 
many different values, as well. This implies that the width w must be large enough to fulfill 2^' > 2*'^', 
from which w > rk(B) > mr(A/r ) follows. D 

Lemma 2.2. Every depth-2 circuit F computing a linear operator can be transformed into an equiv- 
alent linear depth-2 circuit of the same degree and width at most mr(Af ). 

Together with Lemma l2n this implies that width(F) = mr(Af ) for every optimal linear depth-2 
circuit F . 

Proof. Let x — )• Mx be the operator computed by F , and let A = Af be the (0, 1 , *)-matrix of F. We can 
construct the desired linear depth-2 circuit computing Mx as follows. Take a completion B of A with 
rk(S) = mr(A). By the definition of completions, the /th row bi of B has the form bt = ai + pj, where 
a, is the /th row of A with all stars set to 0, and p,- is a (0, l)-vector having no I's in positions, where 
this row of A has non-stars. The /th row m,- of the original (0, 1 )-matrix M is of the form m, = a, + m'^, 
where m\ is a (0, l)-vector which coincides with m,- in all positions, where the /th row of A has stars, 
and has O's elsewhere. 

The matrix B has r = rk(B) = mr(A) linearly independent rows. Assume w.l.o.g. that these are 
the first rows bi,... ,br of B, and add r linear gates computing the scalar products {bi,x),. . . , {b,-,x) 
over GF2 on the middle layer. Connect by wires each of these linear gates with all input and all output 
nodes. Note that the /th output gate, knowing the vectors p, and m'^, can compute both scalar products 
{Pi,x) and {m'i,x) by only using existing direct wires from inputs xi,... ,Xn to this gate. Hence, using 
the r linear gates {b\,x),... ,{br,x) on the middle layer, the /th output gate, for / < r, can also compute 
the whole scalar product {mi,x) of the input vector with the /th row of M by: 

(m;,x) = (a,-,x) © {m'j,x) = {bi,x) © {Pi,x) © (m-,x) . 

For / > r, just replace vector bi in this expression by the corresponding linear combination ofbi,...,br. 
We have thus constructed an equivalent linear depth-2 circuit of the same degree and of width r = 
mr(Af). D 

By Lemma [2!2l the main question is: How much the width of a circuit F can be smaller than the 
min-rank of its matrix Afl Ideally, we would like to have that width (F) > e • mr(Af ): then the width 
of the resulting linear circuit would be at most l/e times larger than that of the original circuit F. 

Lemma IZT] lower bounds the width of linear circuits F in terms of the min-rank of their (0,1,*)- 
matrices Af. We now show that the Min-Rank Conjecture implies a similar fact also for general 
(non-linear) circuits. 



Lemma 2.3. For every depth-2 circuit F computing a linear operator in n variables, we have that 

width (F) >n — log2opt(A/7) . 

Hence, the Min-Rank Conjecture (stating that opt(A) < 2"-i^MA)^ implies that width(F) > e ■ 
mr{Af). 

Proof. Let M be an m-by-n (0, l)-matrix. Take a depth-2 circuit F of width w computing Mx, and 
let Af be the corresponding (0, 1 , *)-matrix. Let H = {hi,... , h„) be an operator computed at the 
gates on the middle layer, and G = {gi,... ,g,n) an operator computed at the gates on the output layer. 
Hence, Mx = G{x,H{x)) for all x G {0, 1}". Fix a vector b e {0, 1}"' for which the set L = {x G 
{0, 1}" : Mx = G{x,b)} is the largest one; hence, \L\ > 2""'''. Note that the operator G'{x) := G{x,b) 
must be consistent with A: its ith component g'i{x) can only depend on input variables Xj to which the 
/th output gate gi is connected. Hence, L is a solution for Af, implying that opt(Af ) > \L\ > 2"^"' 
from which the desired lower bound w >n — log2 opt (A/r) on the width of F follows. D 

We can now show that the Min-Rank Conjecture (Conjecture O indeed implies the Linearization 
Conjecture (Conjecture [T])- 

Corollary 2.4. Conjecture\2\implies Conjecture\l\ 

Proof. Let F be a depth-2 circuit computing a linear operator in n variables. Assuming Conjecture |2j 
Lemma 1231 implies that e •mr(Af) < n — log2opt(Af ) < width(f ). By Lemma l2!2l the circuit F 
can be transformed into an equivalent linear depth-2 circuit of the same degree and width at most 
mr(A/r) < width (F)/£. D 

Hence, together with Valiant's result, the Min-Rank Conjecture implies that a linear operator Mx 
requires a super-linear number of gates in any log-depth circuit over {&, V, -■}, if every depth-2 circuit 
for Mx over {©, 1} of width w = 0{n/\n\nn) requires degree n^^^\ 

Finally, let us show that the only "sorrow", when trying to linearize a depth-2 circuit, is the 
possible non-linearity of output gates — non-linearity of gates on the middle layer is no problem. 

Lemma 2.5. Let F be a depth-2 circuit computing a linear operator If all gates on the output layer 
are linear boolean functions, then F can be transformed into an equivalent linear depth-2 circuit of 
the same degree and width. 

Proof. Let M be an m-hy-n (0, 1) -matrix, and let F be a depth-2 circuit of width w computing Mx. 
Let // = (/ii , . . . , /itv) be the operator // : {0, 1 }" — ^ {0, 1 }^ computed by the gates on the middle layer. 
Assume that all output gates of F are linear boolean functions. Let B be the m-hy-n adjacency (0, 1)- 
matrix of the bipartite graph formed by the direct input-output wires, and C be the m-hy-w adjacency 
(0, l)-matrix of the bipartite graph formed by the wires joining the gates on the middle layer with 
those on the output layer. Then 

Mx = Bx®C-H{x) for all ;cG {0,1}", 

where C •H{x) is the product of the matrix C with the vector y = H{x). Hence, 

CH{x)=Dx (4) 

is a linear operator with D = M®B. Write each vector x = {x\,... ,Xn) as the linear combination 

n 

x = Y,Xiei (5) 



of unit vectors ^i, . . . ,e„ G {0, 1}", and replace the operator H computed on the middle layer by a 

linear operator 

n 

H'{x) := Y^XiH{ei) (mod 2) . (6) 

1=1 

Then, using the linearity of the matrix- vector product, we obtain that (with all sums mod 2): 



C-H{x)=D-[yxiei\ by © and © 

= ^XjDei linearity 

= £x,C-//(^;) by© 

= C • ( ^XiH{ei) j linearity 

= C-H'{x) by®. 

Hence, we again have that Mx = Bx@CH' {x) , meaning that the obtained linear circuit computes the 
same linear operator Mx. D 

3 Bounds on opt (A) 

Recall that opt(A) is the largest possible number of vectors in a solution for a given (0, 1, *)-matrix A. 
The simplest properties of this parameter are summarized in the following 

Lemma 3.1. Let A be an m-by-n (0, l,*)-mafnx If A' is obtained by removing some rows of A, then 
opt(A') > opt(A). If A = [B,C\ where B is an m-by-p submatrix of A for some I < p <n, then 



opt(B) • opt(C) < opt(A) < opt(B) • 2"-P . 

Proof. The first claim opt(A') > opt(A) is obvious, since addition of new equations can only decrease 
the number of solutions in any system of equations. 

To prove opt (A) < opt(B) • 2"^^, take an optimal solution La = {x: Mx = G{x)} for A; hence, 
\La\ = opt(A). Fix a vector b G {0, 1}"^^ for which the set 

LB = {ye{0,l}P: {y,b)GLA} 

is the largest one; hence, \Lb\ > opt(A)/2""P. The completion M of A has the form M = [M',M"], 
where M' is a completion of B and M" is a completion of C. If we define an operator G' : {0, 1}^ — )• 
{0,1}™ by 

G'{y):=Giy,b)(BM"b, 

then M'y = G'{y) for all y G Lg. Hence, Lb is a solution for B, implying that opt(A) < \Lb\ ■ 2""'^ < 
opt(B)-2"-P. 

To prove opt(A) > opt(B) • opt(C), let Lb = {y e {0, 1}^ : M'y = G'{y)} be an optimal solution 
for B, and let Lc = {z ^ {0, 1}""^ : M"z. = G" {z)} be an optimal solution for C. For any pair x = 
{y,z) eLgX Lc, we have that Mx = G{x), where M = [M',M"] and G{y,z) := ^'(3;) G"(z). Hence, 
the set Lb X Lc '^ {0; 1}" is a solution for A, implying that opt(B) • opt(C) = \Lb x Lc\ < opt(A), as 
claimed. D 



Let A be an m-by-n (0,1,*) -matrix. The min-rank conjecture claims that the largest number 
opt(A) of vectors in a solution for A can be upper bounded in terms of the min-rank of A as opt(A) < 
2n-emr(/i) rpj^g claim is true if the min-rank of A is "witnessed" by some (0, l)-submatrix of A, that 
is, if A contains a (0, l)-submatrix of rank equal to the min-rank of A. This is a direct consequence of 
the following simple 

Lemma 3.2. If A is an m-by-n (0, \,*)-matrix, then opt(A) < 2"^^'^^^' for every {0,\)-submatrix B of 
A. 

Proof. Let B be a p-hy-q (0, l)-submatrix of A. Since B has no stars, only constant operators can be 
consistent with B. Hence, if L C {0, 1}^ is a solution for B, then there must be a vector b G {0, 1}'' 
such that Bx = b for all x & L. This implies \L\ < 2'?^*(^). Together with Lemma [TTl this yields 
opt(A) < 2''-*(^) • 2"^^ = 2''-*(^). D 

The max-rank Mr(A) of a (0, 1, *)-matrix A is a maximal possible rank of its completion. A line 
of A is either its row or its column. A cover of A is a set X of its lines covering all stars. Let cov(A) 
denote the smallest possible number of lines in a cover of A. 

Lemma 3.3. For every m-by-n (0, 1, *)-mafn'x A, we have that 

opt(A) < 2"-^^'-(^)+™^('4) . 

Proof. Given a cover X of the stars in A by lines, remove all these lines, and let Ax be the resulting 
(0, l)-submatrix of A. Clearly, we have: Mr(A) < rk(Ax) + \X\. (In fact, it is shown in Q that 
Mr(A) = minx (rk(Ax) + \X\), where the minimum is over all covers X of A.) Take a cover X of A of 
size \X\ = cov(A). Hence, Mr(A) < rk(Ax) +cov(A). Since Ax is a (0, l)-submatrix of A, Lemma [3^ 
yields opt(A) < 2"~'^^^''\ where rk(Ax) > Mr(A) - \X\ = Mr(A) - cov(A). D 

Given a depth-2 circuit F , let m(F) denote the largest number of wires in a matching formed by 
direct input-output wires. That is, m(F) is the largest number of *-entries in the matrix Ap of F, no 
two on the same line. By the well-known Konig-Egevary theorem, stating that the size of a largest 
matching in a bipartite graph is equal to the smallest set of vertices which together touch every edge, 
we have that m(A) = cov(A/r ). This leads to the following 

Corollary 3.4. Every depth-2 circuit F computing a linear operator can be transformed into an 
equivalent linear depth-2 circuit F' of the same degree and 

width(F') < width(F) + m(F) . 

Proof. Let Ap be the (0, 1 , *)-matrix of F. By Lemmas [2.31 and [331 we have that 

width(F) >n- logj opt(Af) >n-[n- Mr(Af) + cov(Af)] 
= Mr(Af ) - cov(Af ) = Mt{Af) - m{F) . 

By Lemma [Z21 the circuit F can be transformed into an equivalent linear depth-2 circuit of the same 
degree and width at most mi{Af) < Mr(Af ) < width(F) + m(F). D 



4 Row and column min-rank 

We are now going to show that the min-rank conjecture holds for stronger versions of min-rank — ^row 
min-rank and column min-rank. 

If A is a (0, 1 , *)-matrix of min-rank r then, for every assignment of constants to stars, the resulting 
(0, l)-matrix will have r linearly independent columns as well as r linearly independent rows. How- 
ever, for different assignments these columns/rows may be different. It is natural to ask whether the 
min-rank conjecture is true if the matrix A has r columns (or r rows) that remain linearly independent 
under any assignment of constants to stars? 

Namely, say that (0, 1, *)-vectors are dependent if they can be made linearly dependent over GF2 
by setting their *-entries to a constants and 1; otherwise, the vectors are independent. 

Remark 4.1. The dependence of (0, 1, *)-vectors can be defined by adding to {0, 1} a new element * 
satisfying a©* = *©a = * for a G {0, 1,*}. Then a set of (0, l,*)-vectors is dependent iff some its 
subset sums up to a (0,*)-vector. Indeed, if some subset sums up to a (0,*)-vector, then we can set 
the *-entries to constants so that the corresponding subset of (0, l)-vectors will sum up (over GF2) to 
an all-0 vector. On the other hand, if no subset sums up to a (0, *)-vector, for every subset, there must 
be a position in which all vectors in this subset have no stars, and the sum of these positions over GF2 
isl. 

Remark 4.2. A basic fact of Linear Algebra, leading to the Gauss-Algorithm, is that linear inde- 
pendence of vectors x,y E {0, 1}" implies that the vectors x + y and y are linear independent as well. 
For (0, l,*)-vectors this does not hold anymore. Take, for example, x = (0, 1) and y = (1,*). Then 
x®y = (1,*) =y. 

For a (0, l,*)-matrix A, define its column min-rank, mrcoi(A), as the maximum number of inde- 
pendent columns, and its row min-rank, mrrow(^). as the maximum number of independent rows. In 
particular, both mrrow(A) and mrcoi(A) are at least r if A contains an r x r "triangular" submatrix, that 
is, a submatrix with zeroes below (or above) the diagonal and ones on the diagonal: 
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where ® G {0, 1, *}. It is clear that neither mrcoi(A) nor mrrow(A) can exceed the min-rank of A. Later 
(Lemma [84] below) we will give an example of a matrix A where both mrcoi(A) and mrrow(^) are by 
a logarithmic factor smaller than mr(A). The question about a more precise relation between these 
parameters remains open (see Problem 1931) . 

Albeit for (0, 1) -matrices we always have that their row-rank coincides with column-rank, for 
(0, l,*)-matrices this is no more true. In particular, for some (0, 1,*) -matrices A, we have that 
mrrow(A) /mrcoi(A). 

Example 4.3. Consider the following (0, 1, *)-matrix: 




Then mrrowC^) = mr(A) = 3 but mrcoi(A) = 2. To see that mrj-owC^) = 3, just observe that the rows 
cannot be made linearly dependent by setting the stars to or 1 : the sum of all three vectors is not 
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a {0,*}-vector because of the 1st column, and the pairwise sums are not {0,*}-vectors because, for 
each pair of rows there is a column containing and 1. To see that mrcoi(A) = 2, observe that the last 
three columns are dependent (each row has a star). Moreover, for every pair of these columns, there is 
an assignment of constants to stars such that either the resulting (0, l)-columns are equal or their sum 
equals the first column. 

We first show that the min-rank conjecture holds with "min-rank" replaced by "column min-rank". 

Theorem 4.4 (Column min-rank). Let Abe a (0,1, *)-matrix with n columns and of column min-rank 
r. Then opt(A) < 2""'". 

Proof. Any m-hy-n (0, l,*)-matrix B of column min-rank r must contain an m x r submatrix A of 
min-rank r. Since opt(B) < opt(A) • 2"^'' (Lemma [3. lb . it is enough to show that opt(A) < 1 for all 
m-hy-r (0, 1, *)-matrices A of min-rank r. 

To do this, let L be a solution for A. Then there is an operator G={g\,... ,gm) '■ {0, 1}'' — ;• {0,1}'" 
such that G is consistent with A and {ai,x) = gi{x) holds for all ;c e L and all / = 1, . . . ,m. Here 
ai ,...,«,„ are the rows of A with all stars set to 0. 

For the sake of contradiction, assume that \L\ > 2 and fix any two vectors x ^y £ L. Our goal is 
to construct a vector c G {0, 1}'" and a completion M of A such that Mx = My = c. Since M must have 
rank r, this will give the desired contradiction, because at most 2''^*('*^) = 2° = 1 vectors z can satisfy 
Mz = c. 

If M is a completion of A = (aij), then its ith row must have the form m,- = a, © pj where pj G 
{0, 1}" is some vector with no I's in positions where the ith row of A has no stars. To construct the 
desired vector Pj for each / € [m], we consider two possible cases. (Recall that the vectors x and y are 
fixed.) 

Case 1: {ai,x) = {ai,y). In this case we can take pj = and c,- = {ai,x). Then {mi,x) = {mi,y) = 
{ai,x) = Ci, as desired. 

Case 2: (a;,jc) / {ai,y)- In this case we have that gi{x) ^ giiy), that is, the vectors x and y must 
differ in some position j where the /th row of A has a star. Then we can take p,- := Cj (the jth unit 
vector) and c, := (a;,x) ®Xj. With this choice of p,-, we again have 

(m;,x) = ((3;,x) © (/?,-, .x) = (a,-,x) © (ey,x) = (a,-,x) ©xy = c,- 

and, since (a;,x) / (fli^y) andxy i^yj, 

{mi,y) = {ai,y) © {pi,y) = {ai,y) © {ej,y) = {ai,x) ®Xj =Ci. D 

Example 4.5. It is not difficult to verify that, for the (0, l,*)-matrix A given by ^, we have that 
nircoi(A) = mr(A) = 2. Hence, no linear solution of the system of semi-lineai" equations Q can have 
more than Un(A) = 2^^^ = 32 vectors. Theorem 14.41 implies that, in fact, no solution can have more 
than this number of vectors. 

The situation with row min-rank is more complicated. In this case we are only able to prove an 
upper bound opt(A) < 2"^'" under an additional restriction that the star-positions in the rows of A form 
a chain under set-inclusion. 

Recall that (0, 1, *)-vectors are independent if they cannot be made linearly dependent over GF2 
by setting stars to constants. The row min-rank of a (0, l,*)-matrix is the largest number r of its 
independent rows. Since adding new rows can only decrease opt(A), it is enough to consider r-by-n 
(0, 1, *)-matrices A with mr(A) = r. 
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If r = 1, that is, if A consists of just one row, then opt(A) < 2"^^ = 2"^'' holds. Indeed, since 
mr(A) = 1, this row cannot be a (0,*)-row. So, there must be at least one 1 in, say, the 1st position. 
Let La = {x: {ai,x) = gi{x)} be a solution for A, where ai is the row of A with all stars set to 0. 
Take the unit vector ei = (1,0, ... ,0) and split the vectors in {0, 1}" into 2"^^ pairs {x,x(Bei}. Since 
the boolean function gj cannot depend on the first variable xi, we have that gi{x(Bei) = gi{x). But 
(a,-,x©ei) = {ai,x) ® 1 / {ai,x). Hence, at most one of the two vectors x and x©ei from each pair 
{x,x(Bei} can lie in La, implying that \La\ < 2"^^. 

To extend this argument for matrices with more rows, we need the following definition. Let 
A = (aij) be an r-by-« (0, l,*)-matrix, and ai,...,^^ be the rows of A with all stars set to 0. Let 
Si = {j: Qij = *} be the set of star-positions in the /th row of A. It will be convenient to describe the 
star-positions by diagonal matrices. Namely, let D,- be the incidence matrix of stars in the /th row of 
A. That is, D, is a diagonal n-hy-n (0, l)-matrix whose jth diagonal entry is 1 iff j G Si. In particular, 
DiX = means that Xj = for all j G 5,-. 

Definition 4.6. A matrix A is isolated if there exist vectors zi , . . . ,2^ G {0, 1}" such that, for all 1 < 
/ < r, we have Dm = and 

/ \ /^ ifi = '; 

{aj,Zi) = < _ .. .. 
[0 ifj<J. 

If Diz; = . . . = DiZi = 0, then the matrix is strongly isolated. 

Lemma 4.7. If A is a strongly isolated r-by-n (0, \,*)-matrix, then opt(A) < 2"^''. 

Proof. Let ai , . . . , a^ be the rows of A with all stars set to 0. We prove the lemma by induction on r. 
The basis case r = 1 is already proved above. For the induction step r — 1 1— )■ r, let 

LA = {xe {0, 1}" : (a,-,x) = gi{x) for all /= 1, ... , r} 

be an optimal solution for A, and let B be a submatrix of A consisting of its first r — 1 rows. Then 

LB = {xe {0, 1}" : (a,-,x) = gi{x) for all / = 1, . . . ,r- 1} 

is a solution for B. Since A is strongly isolated, the matrix B is strongly isolated as well. The induction 
hypothesis implies that \Lb\ < 2"^^''^^'. 

Let z = Zr be the r-th isolating vector. For each row / = 1 , . . . , r — 1, the conditions (z, a;) = and 
DiZ = imply that ((x©z),a,-) = (x,a,-) and gi{x®z) = gi{x). That is, 

X G Lg iff x©z G Lb. 

For the rth row, the conditions {z,ar) = 1 and DrZ = imply that ((x©z),ar) / (jc,«r) whereas 

gr{x®z) =gr{x). That is, 

xGLa iff x®z<^ La. 

Hence, for every vector x ^Lb, only one of the vectors x and x © z can belong to La, implying that 

opt(A) = |LA| <|L^|/2<2"-^ D 

We are now going to show that (0, 1, *)-matrices with some conditions on the distribution of stars 
in them are strongly isolated. For this, we need the following two facts. A projection of a vector 
x = (xi , . . . ,x„) onto a set of positions / = {/i , . . . , 4} is the vector 

X\l — \Xi\ ) • • • ^Xii^) . 

A (0, 1 , *)-vector x is independent of (0, 1 , *)-vectors y^,. .. ,y^_ if no completion of x can be written 
as a linear combination of some completions of these vectors. 
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Lemma 4.8. Let x,y j , . . . , j^ Z>e (0, 1 , *)-vectors, and I = {i: x,- 7^ *}. Ifx is independent ofji ,. .. ,yj^, 
then X \j is also independent ofy^ |"/, . . . , j;;, [/. 

Proof. Assume that x \i is dependent on the projections y^ \u- ■ ■ ^y^ \i- Then there is an assignment of 
stars to constants in the vectors y,- such that xf/ can be written as a linear combination of the projections 
y'l \h-- ■ ^y'k \i o'^ ^ of '^he resuhing (0, l)-vectors /j , . . . ^y'^. But since x has stai^s in all positions outside 
/, these stars can be set to appropriate constants so that the resulting (0, l)-vector x' will be a linear 
combination of 3^'^ , . . . , j^, a contradiction. D 

Lemma 4.9. Let a € {0, 1}" he a vector and M he an m-hy-n (0, \)-matrix of rank r <n—\. If a is 
linearly independent of the rows ofM, then there exists asetZCL {0, 1}" of \Z\ > 2"^''^' vectors such 
that, for all z(z Z, we have {z.,a) = 1 and Mz = 0. 

Proof. Let Z= {z: Mz = 0, (a , z) = 1 }, and let M' be the matrix M with an additional row a. Note that 
Z = ker(M) \ker(M'), where ker(M) = {z: Mz = 0} is the kernel of M. Since rk(M') = rk(M) + l<n, 
we have that |ker(M')| = |ker(M)|/2, implying that 

\Z\ = |ker(M) \ker(M')| = |ker(M)|/2 > 2""''"^ . D 

Lemma 4.10. If A is an r-by-n (0, \,*)-matrix with mr(A) = r, then A is isolated. 

Proof. Let ai, . . . ,a,. be the rows of A with all stars set to 0. Let / C {1, ...,«} be the set of all 
star-free positions in the ith row of A, and consider an (r— l)-by-|/| (0, l)-matrix M,- whose rows 
are the projections a'- = aj \j of vectors aj with j / / onto the set /. By Lemma 1481 the projection 
'^'i = ^i \i of the /th vector at onto / cannot be written as a linear combination of the rows of M,-; 
hence, rk(M;) < |/| - 1. Since 2\'\-MM,)-i > 20 = l, Lemma 1431 gives us a vector z^ G {0, 1}I^I 
such that {z'i,a'i) = 1 and {z'i,a'j) = for all j / /. But then Zi '■= (z;,0) is the desired (0, l)-vector: 
DiZi = D; • = 0, {zi,ai) = {z'i,a'j) = 1, and {zi,aj) = {z'i^a'j) = for all rows j / /. D 

Say that an r-by-n (0, 1 , *)-matrix A is star-monotone if the sets ^i , . . . , 5,- of star-positions in its 
rows form a chain, that is, if ^i C ^2 ^ . . . C Sr. 

Theorem 4.11 (Star-monotone matrices). Let A be a (0, l,*)-matrix with n columns. If A contains an 
r-by-n star-monotone submatrix ofmin-rank r, then opt(A) < 2"^''. 

Proof. Since addition of new rows can only decrease the size of a solution, we can assume that A itself 
is an r-by-?i star-monotone matrix of min-rank r. Let ai, . . . ,a^ be the rows of A with all stars set to 
0. By Lemma l4.10[ the matrix A is isolated. That is, there exist vectors zi,---,Zr ^ {0,1}" such that: 
{ai,Zj) = 1 iff / = j, and DiZi = for all 1 < / < r. Since Sj C Si for all j < i, this last condition implies 
that DjZi = for all I < j < i < r, that is, A is strongly isolated. Hence, we can apply Lemma l477l D 

5 Solutions as independent sets in Cayley graphs 

Let A = (aij) be an m-hy-n (0, 1 , *)-matrix. In the definition of solutions L for A we take a completion 
M of A and an operator G{x), and require that Mx = G{x) for all x G L. The operator G = {gi,...,gfn) 
can be arbitrary — the only restriction is that its ith component gi can only depend on variables corre- 
sponding to stars in the ith row of A. In this section we show that the actual form of operators G can 
be ignored — only star-positions are important. To do this, we associate with A the following set of 
"forbidden" vectors: 

Ka = {x£ {0, 1}" : 3/ G [m] Ax = and (a,-,x) = 1} , 
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where D,- is the incidence n-hy-n (0, l)-matrix of stars in the /th row of A, and a,- is the /th row of A 
with all stars set to 0. Hence, Ka is a union Ka = U^i ^; of ^ affine spaces 

Lemma 5.1. For every vector x £ {0, 1}", x£ Ka if and only ifMx ^ Ofor all completions M of A. 

Proof. {=>): Take a vector x G Ka- Then there exists an / € [m] such that vector x has zeroes in all 
positions, where the /th row of A has stars, and {aj,x) = 1, where a,- is obtained by setting all stars in 
this row to 0. So, if bi is any completion of the /th row of A then {bi,x) = {ai,x) = 1. Thus, the scalar 
product of X with the /th row of any completion of A must be equal to 1. 

(<^): Take a vector x^Ka- We have to show that then Mx = for at least one completion M of A. 
The fact that x does not belong to Ka means that for each / G [m] either (i) (a,-,x) = 0, or (ii) (fl;,x) = 1 
but vector x has a 1 in some position j, where the /th row of A has a star. We can therefore construct 
the /th row m,- of the desired completion M of A with Mx = by taking m,- = a,-, if (i), and m,- = a, + ej, 
if (ii). In both cases we have {mi,x) = 0, as desired. D 

The sum-set of two sets of vectors S,T CI {0, 1}" is the set of vectors 

S + T = {x®y: x£Sandy£T}. 

Theorem 5.2. Aset LC {0, 1}" is a solution for A if and only if {L + L)r\KA = 0. 

Proof. Observe that the sum x © y of two vectors belongs to Ka iff these vectors coincide on all stars 
of at least one row of A such that {ai,x) ^ {ai,y). By this observation, we see that the condition 
(L + L) n iCi = is equivalent to: 

\/x,y £ L V/ G [m] : DiX = Diy implies (a,-,x) = {ai,y). (7) 



Having made this observation, we now turn to the actual proof of Theorem [ 

(=^) Let L be a solution for A. Hence, there is an operator G = {gi,...,gm) consistent with A such 
that (a,-,x) = gi{x) for all x G L and all rows / G [m\. To show that then L must satisfy ^, take any 
two vectors x,y € L and assume that D,x = Diy. This means that vectors x and y must coincide in all 
positions where the /th row of A has stars. Since gi can only depend on these positions, this implies 
gi{x) = gi{y), and hence, (a;,x) = {ai,y). 

(<^) Assume that L C {0, 1}" satisfies ^. We have to show that then there exists an operator 
G = {gi,---,gm) consistent with A such that (a,-,x) = g,(x) for all x G L and / G [m]; here, as before, a,- 
is the /th row of A with all stars set to 0. The /th row of A splits the set L into two subsets 

L^ = {x e L: {ai,x) = 0} and L] = {x eL: {ai,x) = 1} . 

Condition ^ implies that D;x y^ Dty for all (x,y) ^ L^ x L\. That is, if St is the set of star-positions in 
the /th row of A, then the projections x f^^ of vectors x in L? onto these positions must be different from 
all the projections y fj. of vectors y in L]. Hence, we can find a boolean function gi : {0, 1}"^' — )• {0, 1} 
taking different values on these two sets of projections. This function will then satisfy gi{x) = (a;,x) 
for all X G L. D 

A coset of a set of vectors L C {0,1}" is a set v + L = {v©x: x CL L] with v G {0,1}". Since 
(v + L) + (v + L) = L + L, Theorem [5]2] implies: 
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Corollary 5.3. Every coset of a solution for a (0, \^*)-matrix A is also a solution for A. 

Remark 5.4. A Cayley graph over the Abelian group ({0, 1}",©) generated by a set A' C {0, 1}" 
of vectors has all vectors in {0, 1}" as vertices, and two vectors x and y are joined by an edge iff 
x®y G K. Theorem l5.2l shows that solutions for a (0, 1, *)-matrix A are precisely the independent sets 
in a Cayley graph generated by a special set Ka- 

Remark 5.5. If A is an m-hy-n (0, l)-matrix, that is, has no stars at all, then Ka = {x: Ax^ 0}. Hence, 
in this case, a set L C {0, 1}" is a solution for A iff there is a vector b € {0, 1}'" such that Ax = b for 
all x&L. That is, in this case, ker(A) = {x: Ax = 0} is an optimal solution. 

6 Structure of linear solutions 

By Theorem |5.2[ a set of vectors L C {0, 1}" is a solution for an m-by-« (0, 1 , *)-matrix A if and only if 
{L+L)r\KA = 0, where Ka C {0, 1}" is the set of "forbidden" vectors for A. Thus, linear solutions are 
precisely vector subspaces of {0, 1}" avoiding the set Ka- Which subspaces these are? We will show 
(Theorem 16.21 ) that these are precisely the subspaces lying entirely in the kernel of some completion 
ofA. 

Each vector subspace of {0, 1}" is a kernel ker(//) = {x : Hx = 0} of some (0, l)-matrix H. Hence, 
linear solutions for A are given by matrices H such that Hx / for all x G /^i; in this case we also say 
that the matrix H separates Ka from zero. By the span-matrix of a (0, l)-matrix H we will mean the 
matrix H whose rows are all linear combinations of the rows of H. 

Lemma 6.1. Let A be a (0, \^*)-matrix and H be (0, \)-matrix. Then ker(//) is a solution for A iffH 
contains a completion of A. 

Proof. To prove (<^), suppose that some completion M of A is a submatrix of H. Let x G Ka- By 
Lemma [5?T1 we know that then Mx / 0, and hence, also Hx / 0. Since Hx = would imply Hx = 0, 
we also have that Hx / 0. 

To prove (=^), suppose that ker(//) is a solution for A, that is, Hx ^ for all x G Ka- Then, for 
every row / G [m] and every vector x G {0, 1}", Hx = and D,x = imply that (a,-,x) = 0. This means 
that a,- must be a linear combination of rows of H and D,. Hence, for each /, the vector a, must lie 
in the vector space spanned by the rows of H and £),-, that is, a, = aj H © j8, D, for some vectors a, 
and j8;. In other words, the /th linear combination ajH of the rows of H is the /th row a, © jS, D, of a 
particular completion M of A, implying that M is a submatrix of H, as desired. D 

Theorem 6.2. Let A be a (0, \,*)-matrix. A linear subspace is a solution for A if and only if it is 
contained in a kernel of some completion of A. 

Proof. (<^): If a linear subspace L C {0, 1}" lies in a kernel of some completion of A then LPi^a = 0. 
by Lemma [5TT] Since L-\-L = L, the set L must be a solution for A, by Theorem l5.2l 

(=^): Let L C {0,1}" be an arbitrary linear solution for A. Then L + L = L and LHKa = 0. 
Take a (0, l)-matrix H with L = ker(//). Since ker(//) HKa = ®, the matrix H separates Ka from 
zero. Lemma [6TT] implies that then H must contain some completion M of A. But then L = ker(//) = 
ker(//) C ker(M), as claimed. D 

Corollary 6.3. For any (0, l,*)-matrix A we have that lin(A) = 2"-™^W_ 

Proof. By Theorem 16.21 lin(A) is the maximum of |ker(M)| = 2"^*''*^' over all completions M of A. 
Since mr(A) is the minimum of rk(M) over all completions M of A, we are done. D 
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Corollary 6.4 (Alternative definition of min-rank). For every (0, 1 , *)-matrix A we have 

mr(A) = min{rk(//) : Hx / Ofor all x £ Ka} . 

Proof. Let R be the smallest possible rank of a (0, l)-matrix separating Ka from zero. To prove 
mr(A) > R, let M be a completion of A with rk(M) = mr(A). By Lemma [STTl the matrix M separates 
Ka form zero. Hence, R < rk(M) = mr(A). 

To prove mr(A) < R, let // be a (0, l)-matrix such that H separates Ka form zero and rk(//) = R. 
By Lemma [6m the matrix H must contain a completion M of A. Hence, mr(A) < rk(M) < rk(//) = 
rk(//) =R. D 

By Lemma Ism the complement of Ka is the union of kernels ker(M) of all completions M of A. 
So, Theorems I5.2l andl 6.2l imply that a subset L C {0, 1}" is: 

• a solution for A iff L + L C (J {ker (M) : M is a completion of A} ; 

• a linear solution for A iff L C ker(M) for some completion M of A. 

7 Structure of general solutions 

The following theorem says that non-linear solutions must be "very non-linear": they cannot contain 
large linear subspaces. Recall that in Valiant's setting (cf. Lemma II. lb we may assume that each 
row of a (0, l,*)-matrix contains at most s = n stars, where 5 > is an arbitrary small constant. 
Define the co-distance of a vector space as the smallest weight of a non-zero vector in its orthogonal 
complement. 

Theorem 7.1. Let L C {0, 1 }" be a solution for an m-by-n (0, 1 , *)-matrix A, and let s be the maximum 
number of stars in a row of A. IfL contains a subspace of co-distance at least s + l, then L lies in a 
linear solution for A. 



Proof. Since L is a solution for A, VK is a linear solution for A as well. Hence, by Theorem 16.21 W is 
contained in a kernel of some completion M of A. Our goal is to show that then the entire solution L 
must be contained in ker(M). To show this, we will use the following simple fact. 

Claim 7.2. Let W C {0, 1}" be a linear subspace of co-distance at least k+\. Then, for every k- 
element subset S C [n] and for every vector y € {0, 1}", there is a vector x &W such that x ^0 and 

y\s=x\s- 

Proof of Claim. The set of all projections of vectors in W onto S forms a linear subspace. If this 
subspace would be proper, then some non-zero vector, whose support lies in S, would belong to the 
orthogonal complement oiW, a. contradiction. D 

Assume now that L ^ ker(M), and take a vector y G L \ker(M). Since y ^ ker(M), we have that 
{mi,y) = 1 for at least one row m, of M. Let 5 be the set of star-positions in the /th row of A (hence, 
l^l < s), and let a,- be this row of A with all stars set to 0. By Claim l7!2l there must be a vector 
X £W C Lnker(M) with 3; ['5= ;c fs, that is, A(x® j) = 0. But x G ker(M) implies that (m,,;c) = 0. 
Hence, {mi,x(By) = {mi,x) © {mi,y) = {mi,y) = 1. Since the vector a,- can only differ from m,- in star- 
positions of the /th row of A and, due to D,(x© j) = 0, the vector x©j has no I's in these positions, 
we obtain that {ai,x(By) = 1. Hence, the vector x © y belongs to ^^4, a contradiction with x,y £ L. 

This completes the proof of Theorem 17. II D 
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8 Relation to codes 

Let 1 < r < « be integers. A (binary) error-correcting code of minimal distance r + 1 is a set C C 
{0, 1}" of vectors, any two of which differ in at least r + 1 coordinates. A code is linear if it forms a 
linear subspace over GF2. The question on how good linear codes are, when compared to non-linear 
ones, is a classical problem in Coding Theory. We now will show that this is just a special case of a 
more general "opt(A) versus lin(A)" problem for (0, 1,*) -matrices, and that Min-Rank Conjecture in 
this special case holds true. 

An {n,r)-code matrix, or just an r-code matrix if the number n of columns is not important, is a 
(0, 1, *)-matrix with n columns and m= (r + 1) (") rows, each of which consists of « — r stars and at 
most one 0. The matrix is constructed as follows. For every r-element subset S of [n] = {I,... ,n} 
include in A a block of r + 1 rows a with a,- = * for all / 5, a,- G {0, 1 } for all / S S, and | {/ G 5 : a,- = 
0}| < 1. That is, each of these rows has stars outside S and has at most one within S. For r = 3 and 
5 = 11,2,3} such a block looks like 



/I 1 1 * • 


■ *\ 


1 1 * • 


* 


1 1 * • 


* 


\1 1 * • 


■ *) 



A Hamming ball around the all-0 vector is defined by 



Ball(r) = {xe{0,\}":0< \x\ < r} , 



where \x\ =xi-\ h^n is the number of I's in x. 

Observation 8.1. If A is an r-code matrix, then Ka = Ball(r) \ {0}. 

Proof. Observe that no vector xe{0,l}'', ;C7^0 can be orthogonal to all r + 1 vectors l,l©ei,...,l© 
e,- in {0,1}'^ with at most one 0. Indeed, if {x,\) =0 then {x,\® e,) = xi for all / = 1 , . . . , r. By this 
observation, a vector x belongs to Ka iff there is an r-element set S C [n] of positions such that x \si^ 
and X \-g= 0, that is, iff jc 7^ and x G Ball(r). D 

Observation 8.2. If A is an {n,r)-code matrix, then the solutions for A are error-correcting codes of 
minimal distance r-\-\, and linear solutions for A are linear codes. 

Proof We have (L + L) n (Ball(r) \{0}) =0iff |.x©3;| >r+l for all x / j e L, that is, iff every two 
vectors x^y ^L differ in at least r + 1 positions. Hence, every solution for an r-code matrix A is a 
code of minimal distance at least r + 1, and Unear solutions are linear codes. D 

Lemma 8.3. For code matrices, the min-rank conjecture holds with a constant e > 0. 

Proof. LetAbe an (?i,r)-code matrix; hence, Ka =Ball(r)\{0}. Setf := [(r— 1)/2J. Since \x(By\ < 
2t < r for all x,y G Ball(?), the sum of any two vectors x y^y from Ball(f) Ues in Ka, implying that 
Ball(f) is a clique in the Cayley graph generated by Ka. Since, by Remark \5A\ solutions for A are 
independent sets in this graph, and since in any graph the number of its vertices divided by the clique 
number is an upper bound on the size of any independent set, we obtain: 



opt(A)<27|Ball(0|=2"/£ 

' i=0 



(8) 



17 



which is the well-known Hamming bound for codes. On the other hand, Gilbert- Varshamov bound 
says that linear codes in {0, 1}" of dimension k and minimum distance d exist, if 



d-2 

I 

;=0 



E(";')<^' 



n—k 



Hence, 



lin(A)>2y£Q. (9) 

Together with ([8]l, this implies that the inequality ([T]) holds with e about 1/2. D 

The example of code matrices also shows that the gap between min-rank and row/column min- 
rank may be at least logarithmic in n. 

Lemma 8.4. If A is an {n,r)-code matrix, then mr(A) = D.{r\n{n/r)) but mrcoi(A) < r+ 1 and 
mrrow(A) < 2r. 

Proof. To prove mr(A) = Q.{r\n{n / r)) , recall that Ka = Ball(r) \ {0}. Hence, Corollary 16.41 implies 
that mr(A) is the smallest possible rank of a (0, l)-matrix H such that ker(//) n Ball(r) C {0}. On the 
other hand, for any such matrix H, its kernel L = ker(//) is a (linear) code of minimal distance at least 
r+\ containing \L\ = 2"^^^W vectors. Since, by Hamming bound ([8]), no code L of distance at least 
r + 1 can have more than N = 2"/{n/r)'^^''^ vectors, we have that 

rk{H) = n- log2 \L\>n- \0g2N = Q.{r\n{n/r)) . 

To prove that mrcoi(A) < r + 1, suppose that A contains some mxk submatrix B of min-rank k. 
Since all k columns must be independent, at least one row b oi B must be *-free and contain an odd 
number \b\ of I's. But every row of A (and hence, also b) can contain at most one 0, implying that 
\b\ >k— I. Together with |Z7| < r, this implies that k <r-\-l. 

To prove that mriow(^) < 2r, recall that each row of A consists oi n — r stars and at most one 0; 
the remaining r (or r — 1) entries are I's. Suppose now that A contains some set X of |X| = ^+ 1 
independent rows. That is, no subset of these rows can be made linearly dependent by setting *'s to 
or 1. The rows in X must be, in particular, pairwise independent. This, in particular, means that the 
set X can contain at most one row without 0-entries. So, let F C X be a set of |F| = /: rows containing 
0-entries. Take any two rows x^y ^Y with xi = and jj = 0. Since x and y are independent and have 
only * 's or 1 's outside their 0-entries, we have that: / 7^ j and either Xj = 1 or j, = 1 . This implies that 
the total number of I's in the rows of Y must be at least the number (2) of pairs of vectors in Y . So, 
there must exist a row x ^Y with \x\ > {2) /\Y\ = {k — l)/2. Together with \x\ < r— 1, this implies 
that k<2r-\,and thus, that |X| = A: + 1 < 2r. D 

9 Conclusion and open problems 

In this paper we pose a conjecture about systems of semi-linear equations and show its relation to 
proving super-linear lower bounds for log-depth circuits. We then give a support for the conjecture by 
proving that some its weaker versions are true. We also show that solutions are independent sets in 
particular Cayley graphs, thus turning the conjecture in a more general (combinatorial) setting. Using 
this, we prove several structural properties of sets of solutions that might be useful when tackling the 
original conjecture. 
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We defined solutions for a given m-hy-n (0, 1 , *)-matrix A as sets L C {0, 1}" of vectors x satisfying 
a system of equations 

{ui.x) = gi{DiX) i=l,...,m, (10) 

where a, is the /th row of A with all stars replaced by 0, gi is an arbitrary boolean function, and D, is 
a diagonal n-hy-n (0, l)-matrix corresponding to stars in the /th row of A. We have also shown (see 
Remark [J!4b that solutions for A are precisely the independent sets in a Cayley graph over the Abelian 
group ({0, 1}",©) generated by a special set of vectors 

Ka = {x: 3i DiX = and (a,-,x) = 1} . (11) 

The following two questions about possible generalizations of the min-rank conjecture naturally arise: 

1. What if instead of diagonal matrices D, in (ITOl ) we would allow other (0, l)-matrices? 

2. What if instead of special generating sets Ka, defined by (fTTI) . we would allow other generating 
sets? 

The following two examples show that the min-rank conjecture cannot be carried too far: its general- 
ized versions are false. 

Example 9.1 (Bad generating sets K). Let G be a Cayley graph generated by the set K C {0, 1}" of 
all vectors with more than n — 2^/n ones. If L C {0, 1}" consist of all vectors with at most njl — ^n 
ones, then (L + L) n A' = 0, that is, L is an independent set in G of size \L\ > 2"^'^('°s"). But any linear 
independent set l! in G is a vector space of dimension at most n — l^Jn. Hence, |L'| < 2"^^^, and 
the gap |L|/|L'| can be as large as 2^'^'. 

Note, however, that there is a big difference between the set K we constructed and the sets Ka aris- 
ing form (0, 1, *)-matrices A: generating sets Ka must be almost "closed downwards". In particular, if 
x^Ka then all nonzero vectors, obtained from x by flipping some even number of its I's to O's, must 
also belong to Ka- Hence, this example does not refute the min-rank conjecture as such. 

Example 9.2 (Bad matrices D,). Let us now look what happens if we allow the matrices Di,... ,D^ 
in the definition of a system of semi-linear equations (fTOl) be arbitrary n xn (0, l)-matrices. A 
completion M of A can then be defined as a (0, l)-matrix with rows m,- = a, + ajDi. Now define 
mr(A|Di , . . . ,D,.) as the minimal rank of such a completion of A. Observe that this definition coin- 
cides with the "old" min-rank, if we take the D,'s to be the diagonal matrices corresponding the stars 
in the /throw of A. 

However, Example |9. 1 1 shows that the min-rank conjecture is false in this generalized setting. To 
see why, we can define appropriate matrices A,Di , . . . ,Dm such that the corresponding set Ka defined 
by (ITTI ) consists of vectors with more than n — 2^ ones: for an arbitrary vector v with more than 
n — 2^/n ones just define a,- and D,- such that the system D,x = 0, {ai,x) = 1 has v as its only solution. 

Except of the obvious open problem to prove or disprove the linearization conjecture (Conjec- 
ture [U or the min-rank conjecture (Conjecture |2l), there are several more concrete problems. 

We have shown (Lemma [8 .41 ) that the gap between min-rank and row/column min-ranks may be 
as large as Inn. It would be interesting to find (0, 1, *)-matrices A with larger gap. 

Problem 9.3. How large can the gap mr(A)/max{mrcoi(A),mrrow(^)} be? 
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The next question concerns the clique number co{Ga) of (that is, the largest number of vertices 
in) Cayley graphs Ga generated by the sets of the sets Ka ^ {0, 1}" of the form (fTTI ). By Remark [S!4l 
solutions for A are independent sets in this graph. Hence, opt(A) is just the independence number 
cc{Ga) of this graph. Since in any N-veitex graph G we have that G)(G) • a{G) < N, this yields 
opt (A) < 2"/«(Ga). On the other hand, it is easy to see that (o{Ga) < 2***^), where M is a canonical 
completion of A obtained by setting all *'s to 0: If C C {0, 1}" is a clique in Ga, then we must have 
Mx 7^ My for all a; / y G C, because otherwise the vector x (By would not belong to Ka- 

Problem 9.4. Give a lower bound on (o{Ga) in terms ofmin-rank mr(A) of A. 

Finally, it would be interesting to eliminate an annoying requirement in Theorem 14. 1 1 1 that the 
matrix A must be star-monotone. 

Problem 9.5. If A is an r-by-n (0, l,*)-matrix ofmin-rank r, is then opt(A) < 2"^''? 
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Table 1 : This table summarizes the concepts introduced in this paper. Here A is a partially defined 
mxn matrix with entries from {0,1,*}. 



Concept 



Notation Meaning 



Completion of A 




Canonical completion of A 




Min-rank 


mr(A) 


Max-rank 


Mr (A) 


Operator G consistent with A 





Solution for A 



Linear solution for A 



opt(A) 
hn(A) 



Min-Rank Conjecture 

Independence of (0,1,*) 

vectors 

Row min-rank 

Column min-rank 

Incidence matrix of *'s 

Set of forbidden vectors 



mrrow(A) 
mrcoi(A) 
Di 

Ka 



A (0,1) -matrix obtained from A by setting its *- 

entries to and 1 . 

All * -entries of A set to 0. 

Minimal rank over GF2 of a completion of A. 

Maximal rank over GF2 of a completion of A. 

The /th coordinate of G : {0,1}" -^ {0,1}*" can 

only depend on variables corresponding to *- 

entries in the /th row of A. 

A set L C {0, 1}" of the form L = {x:Mx = G{x)}, 

where M is a completion of A, and G is an operator 

consistent with A. 

A solution for A forming a linear subspace of 

{0,1}". 

Maximum size of a solution for A. 

Maximum size of a linear solution for A; lin(A) = 

on— mr(A) 

opt(A) < 2"-'^™''('*) for a constant £ > 0. 

Cannot be made linear dependent by setting *'s to 

constants. 

Maximal number of independent rows. 

Maximal number of independent columns. 

Diagonal (0, l)-matrix with D,[j,j] = 1 iff A[/, j] = 

*. 

All vectors x G {0,1}" such that Dtx = and 

(a,,x) = 1, where ai is the /th row of A with all 

stars set to 0. Main property: L is a solution for A 

iff (L + L)nii:A = 0■ 
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